1. Field of the Invention
This invention relates in general to image processing systems and methods and in particular to a machine which accepts a real time video image in the form of a matrix of picture elements (hereafter called pixels) and remaps such image according to a selectable one of a plurality of mapping functions to create an output matrix of pixels. Such mapping functions, or transformations may be any one of a number of different transformations depending on the objective of the user of the system. More particularly, the invention relates to a system for the generation of an output pixel matrix according to any one of a number of transformation functions in real time so that the input image may be spatially remapped prior to presenting them to an optical correlator. Still more particularly, the invention relates to mapping functions useful to transform the input video image into an output pixel matrix for display on an image display device as an aid to people with low vision problems such as retinitis pigmentosa and maculopathy. In a particular embodiment, a single flow architecture for video rate image transformations is provided.
2. Description of the Prior Art
Optical correlators are used in pattern recognition, attitude estimation systems and the like to process an input image to create an adjusted image for comparison with a standard image. Such optical correlators (particularly phase-only correlators) are excessively sensitive to scale change and rotation of the input image. Much effort must be expanded in designing and constructing filters that ameliorate such sensitivity.
Mapping or transforming of an input image to produce rotation and scale invariance in an output image while still in the spatial domain substantially eliminates the rotation and scale sensitivity requirements of a correlator of the transformed image. One such mapping or transformation with properties of rotation and scale invariance is disclosed, for example, in U.S. Pat. No. 4,267,573 to Chaikin and Weiman. The Chaikin and Weiman patent describes an image transformation device, which may be a physical device, or a combination of an A/D converter of a video image followed by a mathematical transformation operating on digital signals, which produces transformed output signals (which may, if desired, be viewed as a video image on a television monitor). The output image of the transformed input signal (i.e., the image of the output signals) is "congruent"; that is, the output image is the same size and shape regardless of changes in size of the input image. The output image is also rotationally invariant, that is, its size and shape is the same regardless of rotation of the input image.
The Chaikin and Weiman patent describes mappings from a digitized input Cartesian grid to a digitized output Cartesian grid. The mapping which serves to associate a picture element or cell (herein called a pixel) in the input Cartesian grid through a particular coordinate transformation with the congruent properties described above is EQU W=e.sup.z,
where Z is the complex number representation (x+i y) of the Cartesian coordinates of the input image pixels, and W is the complex number representation of the coordinates of the transformed output image. Geometrically, this mapping or transformation equation is equivalent to providing a Cartesian output matrix of columns and rows where the output rows represent rays or radii centered in the input Cartesian input matrix and the output columns represent rings and circles centered in the Cartesian input matrix. Each ring radius is determined by EQU W=e.sup.x
and the angle of each ray or radii is EQU .theta.=2.pi.y/y.sub.max.
The Chaikin and Weiman patent also describes curvilinear logarithmic spiral mapping systems which have the property of rotational and scale invariance, but the output coordinate system is not the convenient Cartesian or square grid as in the ray/ring transformation described above. In one, the output grid is a skewed x-y Cartesian grid where the mapping is formed by two orthogonally intersecting families of logarithmic spirals. In another, the output grid is a lattice of equilateral triangles where the mapping is formed by two families of intersecting nonorthogonal log spirals.
The Chaikin and Weiman patent is an important contribution to the art of image transformation in the field of pattern recognition. It is only one image transformation solution, however, of the universe of such solutions, to the general problem of preparing an output image for presentation to a pattern recognition system so that a priori known input image in-plane motions (the so-called "optical flow" in the time-varying image) become purely translational in the output image. It is a great advantage to discover transformations which produce such property in the output image, because pattern recognition systems are simplified where only translation of the output image is required to match a standard or known image.
Before the invention described below, there has existed only one image processor or "remapper" that can operate with sufficient speed and flexibility to permit investigating different transformation patterns in real time. "Real time" is usually considered to be about 30 frames per second with minimum lag. Such an image processor is desirable to investigate an optimal transformation for a given image processing problem. For example, where in-plane rotation and scale invariance are important for one problem, the ray/ring transformation of Chaikin and Weiman described above should be considered. Where Forward Looking Infra Red (FLIR) images are to be transformed or remapped prior to pattern recognition processes, a projective transformation or ratio of affines transformation should be considered. Where an input image should be mapped to an output image such that certain information of the input is displaced and enhanced (e.g., magnified, edge enhanced, etc.) in the output image to benefit people with low vision problems, another transformation should be considered, and so on.
Optical correlators and other pattern recognition methods deal well with translation, but are difficult to design where the output image (i.e., the input image to the correlator) it is working on must be rotated or have its scale changed. The Chaikin and Weiman transformations, such as the ray/ring transformation to a Cartesian output grid, achieve such scale or rotation translation with ease, because translation in one Cartesian direction corresponds to a change in scale while translation in the other Cartesian direction corresponds to a change in rotation. But, if the position of the input image must be changed for pattern recognition, optical correlators can not easily be designed to cope with such changes. Accordingly, the image processing art has needed a programmable remapping system to easily and rapidly perform spatial transformation as a front end to an optical correlator.
The embodiment of prior art systems would typically include two parallel processors. The first is a collective processor which maps multiple input pixels into a single output pixel. This processor is used where one or more input pixels are compressed into such single output pixel. In other words, the value of each output pixel may be affected by the values of several input pixels. The second is an interpolative processor where a single input pixel affects a plurality of output pixels. The interpolative processor performs an interpolation among pixels in the input image where a given input pixel may affect the value of many output pixels.